An augmented reality system can insert virtual objects in a user's view of the real world. There may be many components in a typical AR system. These include: data acquisition, data processing, object detection, object tracking, registration, refinement, and rendering components. These components may interact with each other to provide the user a rich AR experience. Several components in detection and tracking in a typical AR system, however, may utilize computationally intensive operations, which can disrupt the AR experience for the user.